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Synopsis 




Modeling in brain science 
Conceptual ("top-down") theory 
A cortical layer 2/3 model 

• Modular network structure 

• Neuron and synapse models 

• Memory storage 

Model output and dynamics 
Ongoing developments 
What about coherence? 
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Modeling in brain science 




STOCKHOLM BRAIN INSTITUTE 



• Massive amounts of data available 

• Molecular, sub-cellular, neuronal, network, macroscopic levels 

• Measurement driven data acquisition 

• Mechanistic understanding still largely lacking. Why? 

• Synthesis needed! Relating data from different sources and levels 
to a coherent picture 

• Mathematical modelling and computer simulation helps to 

• Organize known data in a coherent fashion 

• Promote model driven experimentation 

• Why large-scale simulations? 

• Connect cellular, synaptic and (brain-scale) network levels 

• Data-driven vs theory-driven modeling 
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Some theories of cortical 
perception/memory function 




• Data -> pruning of models, but no consensus 



Synfire chain paradigm 
Hebbian cell assembly theory 

• Attractor network paradigm, Ising model 

Binding by synchrony/phase locking 
Global workspace theory 
Liquid state machine theory 

■ ■ ■ 

All hypothetical, related 

All subject to simulation and analysis 
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Hebbian synapses and cell assemblies 
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Hebb D 0, 1949: The Organization of Behavior 







Bliss and Lomo, 1973 
Levy and Steward, 1978 
LTP/LTD 



• Cell assembly = mental object 

• Gestalt (Holistic) perception 

• Perceptual completion 

• Figure-background segmentation 

• Perceptual rivalry 

• Milner P: + Lateral inhibition 

• After activity « 500 ms 

• Persistent, sustained 

• Fatigue = Adaptation, synaptic depression 

• Generalizes to associative memory 

• Association chains 

• Time-asymmetric synaptic plasticity 
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... implemented as attractor 

networks 
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• e.g. Hopfield network 

• Mathematical instanciations of Hebb's 
theory 

• Recurrently connected 

• Layer 2/3 

• Hebbian learning rule 

• Sparse 

• Connectivity 

• Activity 

• Modular ("Potts network") 

• Extensively studied 

• Simulations, e.g. memory properties 

• Theoretical analysis 

• Efficient content-addressable memory! 
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Criticized on several points 

• Unnatural network structure 

• Dense, symmetric 
connectivity 

• Dense activity 
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Old idea, yes 
But still useful! 




Support for biological plausibility 
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Olfactory cortex = attractor network? 

• Haberly, Chem Senses 2001 
Hippocampus = attractor network? 

• Memory reactivation during sleep 

• Wilson and McNaughton, Science 1994 

• Attractor state like transitions 

• O'Keefe et al., Science 2005 

In vitro (local) UP-states 

• Crossart et al. 2003 

PFC, persistent activity working memory 

• Wang et al. 2001 
Recurrent cortical connectivity 

• L 2/3 & 5, higher order (association) 
cortex 

• Top-down projections 



Can an attractor memory be 
implemented by a network of 
real neurons and synapses? 

• Could it do some memory, 
perception? 

• If so, what relation to cortical 
functional architecture and 
dynamics? 
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Cortical computational unit(s)? 

Neuron, minicolumn, hypercolumn, area, ... 
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Data on cortical 
connectivity, layers, 
columns, etc 

• From anatomy 

• From pairwise recordings 

Computational units? 

• Neuron 

• Anatomical minicolumn 

• Sub-network 

• Species/location differences? 

Hyper/Macrocolumns 
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The layer 2/3 cortex model 

Circuit layout 
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1 17%2.5mV 



2 30%0.30mV 






3 different cell types 

• Pyramidal cells, Basket cells, RSNP cells 

• RSNP = Regular Spiking Non-Pyramidal cells (dendritic targeting, 
vertically projecting, e.g. double bouquet cells) 

Functional minicolumns with 30 pyramidals, 2 RSNP 

Hypercolumns with minicolumns and basket cells 

(Model currently being extended with layers 4 and 5) 
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The layer 2/3 cortex model 

Single cell model 
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Hodgkin-Huxley formalism 

Na, K, K Ca , Ca-channels 

• Ca AP and Ca NMDA pools 

Pyramidal cells 

• 6 compartments 

IS, soma, 1 basal, 3 apikal dendritic 

Inhibitory interneurons 

• 3 compartments 

IS, soma, dendritic 

AP and AHP shapes 

Firing properties, adaptation 

Neuron populations 

• Cell size spread (±10%) 
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The layer 2/3 cortex model 

Synaptic properties and connectivity 
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Local 
RSNP 



Distant 
pyramidal 




Local 
basket cell 



Local 
pyramidal 



Synaptic transmission 

Glutamate (AMPA & voltage dependent NMDA) 
Pyr-pyr synapses depressing 

• GABA A 

No gap junctions 

Synaptic targeting of soma and dendrites 
3D geometry => delays 

• 0.1 - 1m/s conduction speed 

Realistic amplitude of PSP:s in larger network models 



pre 



post 




>Vv_ 



Pyramidal- pyramidal fast synaptic depression 

[Tsodyks, Uziel, Markram 2000] 
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One of the 9 
hypercolumns 



Active 

minicolumn 

(30 pyramidal 

cells) 



Active basket 
cell | 



Active RSNP 
cells I 



Network layout 
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1x1 mm patch 
9 hypercolumns 
Each hypercolumn 

• 100 minicolumns 

• 100 basket cells 
29700 neurons 

15 million synapses 
100 patterns stored 

W created by 
training an 
isomorphic 
population unit 
network 
(asymmetric!) 

W obeys pairwise 

connection 

statistics 
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100 hypercolumns 

Spontaneous activity 





330000 neurons 

161 million 
synapses 



4x4 mm 
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8 rack BG/L simulation 



October 2006 
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22x22 mm cortical patch 

• 22 million cells, 11 billion synapses 
SPLIT simulator by KTH 

8K nodes, co-processor mode 

• used 360 MB memory/node 
Setup time = 6927 s 
Simulation time = 1 s in 5942 s 
Massive amounts of output data 

Network operation similar as smaller 
versions of the model 



77 % estimated speedup Neural simulations scale very 

• Point-point communication slows (?) well! 

• Linear speedup to 4K nodes Today's largest simulations are 

much larger, but simpler 

Djurfeldt M, Lundqvist M, Johansson C, Rehn M, Ekeberg 6, and Lansner A (2008): Brain-scale simulation of 
the neocortex on the Blue Gene/L supercomputer. IBM J R&D 52:31-41 
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Supercomputing for brain 

modelling 
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Real-time human brain size simulatiofi 



1980 



IBM 
BlueGene/L 
128K cores 




Next generation 

supercomputers 

>1M cores 

10 PF 



1990 1995 2000 2005 2010 2015 2020 2025 



year 
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■ 2000+ neurons 

■ 250000+ synapses 

■ 5 s = 600 s on PC 



Spontaneous 
"resting activity 
Tsodyks, Grinvald 
et al. 1999 



ff 



Mishas dynamics 



Stimulus resets! 
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Lundqvist M, Rehn M, Djurfeldt M and Lansner A (2006). Attractor 
dynamics in a modular network model of the neocortex. Network: 
Computation in Neural Systems: 17, 253-276 
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Attractor 
hopping 
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Spike raster 

• 9 attractors 



' switchspike5. out' 
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Tirne-freq blocks of average LFP power spectra 



• LFP power 
spectrogram 




Brain Coordination, Caribbean, March 2010 




Structured vs Unstructured W 
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• Same cell-cell connections, but randomly permuted (right) 

• Pairwise connection statistics obeyed in both cases 

• NOTE: Pairwise statistics not sufficient -> specific activity 
dependent plasticity important ! 



Many E-l cortex models use random connectivity 
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Kl Cortex course 
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Bimodal V 



m 



D 



50 mV 



Is 





-0.08 -0.075 -0.07 -0.065 -0.06 -0.055 -0.05 -0.045 -0.04 

Jeffrey Anderson, Man Lampl, Iva Reichova, Matteo 
Carandini, and David Ferster. Stimulus dependence of 
two-state fluctuations of membrane potential in cat visual 
cortex. Nat. Neurosci., 3(6): 617-621, 2000. 
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Attractor dynamics 

Pattern rivalry, fast "decision" 
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Parameter sensitivity of 

active state 
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Each parameter x2 and x0.5 

• though x1.25 and x0.8 for slow Ca 
dynamics 

• or attractor states were found never 
to terminate 

The qualitative behavior of the 
system is quite robust to parameter 
changes 

• Most sensitive to slow Ca dynamics 

Tests by pharmacological 
manipulation? 
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Biological plausibility of Hebbian cell 
assembly/attractor network model? 

Pattern reconstruction 

• Figure-background 

• Pattern completion 

• Pattern rivalry 

• 50 - 100 ms settling time 

Persistent after-activity 

• 150 ms -2 sec 

• NMDA Ca , K Ca modulation 

Attentional blink model 

Operation robust to parameter changes and 
scaling 



Replicates oscillatory field potentials and spiking 
patterns seen in vivo 
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Bistable, irregular firing, spike 
synchronization 




Ground state 



10/1 Hz 



Active (memory) state 




Foreground neurons 
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Bistable, irregular firing, spike 
synchronization cont'd 

LFP spectrograms 




• Pyr<->Bc gamma oscillator 

• Balanced excitation-inhibition/ 
inhibition dominated 

• LFP oscillations 

• ground state (20-40 Hz) 

• active state (40-60 Hz) 

• not detectable at single cell level 
(spikes) 

• Spiking activity comparable to 
experimental data 

• Joelving et al. 2007 

• Rudolph M, et al. 2007 
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Pyramidal cells 
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Spiking activity 

in ground and active state 
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Ground state - diffuse 
Active state - focused 
V m is oscillatory 

• Foreground neurons lead 

N:o spikes same in ground 
and active states 
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Oscillations and irregular 

spiking 



First attractor memory model to show population 
oscillations in both ground and active state 

• Beta (?) in ground state 

• Gamma in active state 

... and low rate irregular spiking of neurons 
Characteristics of in vivo cortical activity 

Why is this model different? 

• RSNP inhibitory interneuron? No ... 

• Modularity - hypercolumns and minicolumns? 
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Importance of modularity 
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Hyper- 
column 




Basket 
cells 




Minicolumn 

Pyramids 

RSNPs 




Hypercolumns - local inhibition 



No modules - global inhibition 



Stable persistent activity 

Local gamma & spike synchrony 

Global asynchrony 



Unstable persistent activity 
Global gamma & spike synchrony 
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Average V m and spikes* in one 
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V m - spike latency histograms 
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One hypercolumn 



Latencies in WITH IN setup - One Hypercolumn 



10 15 20 25 30 35 40 
latency [ms] 



Latencies in BETWEEN setup - One Hypercolumn 



5 10 15 20 25 30 35 40 

latency [ms] 



Latencies in TOTAL setup - One Hypercolumn 



Four hypercolumns 



Latencies in WITHIN setup - Four Hypercolumn 



5 10 15 20 25 30 35 40 

latency [ms] 




Nine hypercolumns 



Latencies in WITH IN setup - Nine Hypercolumn 
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Importance of modular 
network structure 
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Modular structure - hypercolumns, 
minicolumns 

Basket cell inhibition local to a 
hypercolum 

The E-l network of the hypercolumn 
- source of gamma oscillations 

Gamma synchrony 

• High within module 

• Low between modules 

Non-synchronous spikes contribute 
to stability of active states 




Spike Triggered Average 









within the reference hypercolumn 
outside the reference hypercolumn 
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Next steps 




More complex, brain-scale multi-area cortical models 

Long-range cortico -cortical connectivity 

• Feed-forward, lateral, feedback ("top-down") projections 

• Self-organization, sparse connectivity 

• Learning, localist representations 
Multi-modal integration 

Interacting memory systems, WM ... LTM 
Attentional modulation, effects of task relevance 
Matching to multiple measurements 

• Synthesis of macroscopic measurements 

• LFP, EEG, VSD, Ca-imaging, BOLD, ... 

• Coherence, synchronization, ... 
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What about coherence? 
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• Quasistable attractors - lower dimensional 
manifold, multi-stability 

• Dynamic [perceptual] decision 

• Spontaneous/Stimulus driven activity 

• EEG Microstates = ? 

• Spatial extent 

• Synchrony at lower frequencies (attractor hopping) 

• Coupled areas/attractors, large-scale cortex models 

• Temporal coordination 

• Overlapping, chained, nested, attractors ... 

• Richer spatio-temporal dynamics 
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EC/IP6/FET/FACETS 




EC/I P7/FET/NEUROCHEM 



Acknowledgements 



• Model development and analysis 

• Mikael Lundqvist, PhD student 

• David Silverstein, PhD student 

• Parallel simulation tools 

• Mikael Djurfeldt, postdoc 

• Data analysis 

• Pawel Hermann, postdoc 

STOCKHOLM BRAIN INSTITUTE 

•:• Swedish Foundation for Strategic Research 

•:• Swedish Research Council and VINNOVA 

•:• IBM 

•:• AstraZeneca 




incf 



International Neuroinformatics 
Coordinating Facility 



Brain Coordination, Caribbean, March 2010 



